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A SYiaOLIC LOGIC FOR REPRESENTING LINEAR MODELS 



Charles E. Hall 
Abstract 

A set of symbols is presented along with logical operators which 
represent the possible manipulations of the linear model. The use of these 
symbols and operators is to simplify the representation of analysis of vari- 
ance models; correlation models and factor analysis models. 
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A SYMBOLIC LOGIC FOR REPRESENTING LINEAR MODELS 



Charles E. Hall 

The development of statistical inference from correlation theory to 
multivariate analysis of variance should be familiar to most readers, even 
though it may be beyond the interests of many. Most of this development has 
been concerned with the linear relationships among variables. Techniques of 
solving linear models of the relationships among variables for statistically 
relevant figures are not familiar to many because the general solutions for 
these models involve the techniques of matrix calculus. These techniques 
have become a little better known during recent years due to the increasing 
popularity of multivariate analysis and to the use of computers to generate 
solutions to large classes of statistical procedures. 

The notational system presented here attempts to solve tiree problems; 

(l) a simplified representation of various linear model procedures, (2) an 
improved description of the matrix calculus procedures used to solve linear 
models, and (3) a simplified interface between the statistician and the 
computer program which does the calculations. 

Following the presentation of the symbolism, there is a section showing 
how the symbolism can be used to expose the identity or similarity of various 
linear models. 

A Short History 

Without belaboring the history of calculation procedures, it may be noted 
that the general solution for linear procedures seems to have been devised by 
R. A. Fisher, about 1950 or so, out of Gauss' method of reducing a general 
square matrix to a diagonal matrix. The first reasonable discussion of the 
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calculation technique^ called the "method of fitting constants/' seems to have 
been undertaken by Wilks (1958) in a paper discussing the solution of non- 
orthogonal analysis of variance. An earlier paper by Wilks (1952) on the 
lambda criterion for multivariate distributions provides cues as to how this 
method can be applied to various kinds of problems in correlations and analysis 
of variance. A paper by Vartak (1955) showed how pseudovariables for factorial 
analysis of variance interactions could be generated from Kronecker products of 
main effect pseudovariables ^ thus simplifying the "method of fitting constants" 
as a calculation technique. 

However^ the history of calculation technique has ignored the "method of 
fitting constants" in the main. Instead, special calculation formulae were 
developed suitable to each of several analysis of variance designs. The bulk 
of practicing statisticians were unable to solve linear models unless their 
design conformed to those models for which special calculation formulae had 
been worked out. 

With the advent of computers, special calculating formulae had to be 
abandoned and calculating technique reverted back to the "method of fitting 
constants" with some improvements. What had been too laborious a technique 
to do by hand became the most efficient method to program for a computer (see 
Bock, 1965 ). Currently, the major computer programs for computing analysis 
of variance and correlation analysis [for example, Cramer's MANOVA (Clyde, 
Cramer, & Sherin, I 966 ), Finn's Multivariance ( 1967 ) and Beaton's f 4STAT (1971)^ 
all use this form of calculation. 

With the growth of generalized computer programs and the increasing scope 
of the probleins they solve, it is becoming more and more obvious to 8,11 stat- 
isticians that all linear models are but variation? on a central theme. There 
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is no need for special calculation formulae for apparently diverse designs. 
With the unification of calculation technique and the attendant recognition 
of the unity of design types, there has also become a need for a notation 
system which will serve for all linear models. 

To this point the following rules are proposed. 

General Notation and Rules 

It is general practice in texts on analysis of variance to refer to row 
and column effects of a factorial design as A or B and interaction effects as 
AxB. In the arithmetic of solving linear models, any effect, say A, has as 
many design parameters (pseudovariables, dummy parameters) as it ha.s degrees 
of freedom. For convenience we will call these the A variables; likewise the 
design parameters for the B effect will be called the B variables. The inter- 
action effect AxB is calculated using the Kronecker product of the A variables 
and the B variables; these product variables will be designated AB with no 
intervening symbol to designate interaction terms and Kronecker product of 
main effect design parameters. 

Rule 1 . Single upper -case Roman letters will designate main effect parameters 
of analysis of variance designs and solutions (excepting V and W). 

Rule 2 . Two or more upper-case Roman letters juxtaposed without intervening 
symbols will indicate the Kronecker product of ma.in effect parameters and 
designate the interactions effect parameters in analysis of variance designs 
and solutions (excepting V and W). 

In texts on correlation analysis, the measurements of variables are often 
designated x, y and z. It is also customary in analysis of variance to desig- 
nate the variable which an experiment manipulates as x. In this system V will 
denote continuous variables. 
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Rule All the continuous variables will be designated by the symbol V. 

On occasion it is necessary to subdivide the design parameters for a 
main effect into subsets. Such occasions arise when the experimental treat- 
ments of an analysis of variance involve a control grouri and several treat- 
ments and one wishes to test control against the average o'" treatments and 
treatments among themselves. The subdivis j.ons , partitions, ci‘ the main effect 
A will be labeled Al, A2, etc. The same holds true for siibsets of the continu- 
ous variables. 

Rule_J+. Division of sets of parameters into subsets will be designated by a,n 
upper case Roman letter (not W) followed by a numeral 1, 2, 5, etc. 

It is assumed that subsets will be enumerated ordinallj'’ from 1 to N and 

that the N subdivisions exhaust the parameters designated by the Roman letter. 

Examples: {A1,A2} = A, {V1,V2,V5) = V. 

There are three basic types oi' linear model problems: correlation, analy- 

sis of variance and factor analysis. These three types of models can all be 
written as solutions to determinantal equations, regardless of the number of 
design parameters or continuous variables involved. For correlations we can 
have an equation of the type 

I SS^ - |i SS^ I - 0 (1) 

where denotes sum of squares for hypothesis and denotes sum of squares 
for total. For analysis of variance we can have an equation of the type 

I SS^ - -A SSg I = 0 (2) 

where SS„ - SS„ - SS„ designates the sum of squares for error. For principal 
E T H 

components analysis we have the equation 
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I ss^, - V I 1 = 0 

V7here SS^ is sometimes reduced to correlations. 

Factor analysis may be distinguished from components analysis oy altering 

(5a) to 

1 SS^ - V 1 =0 

where denotes uniqueness variance. In this case it is customary to reduce 

p 

SSr^, to correlations and U correspondingly. The various forms of factor and 
components analysis are usually capable of representa\'.ion as eigenvalue problems 
and therefore have determinantal equations similar to (5a) or (5b) witti only 
minor alterations.^ 

Rule 5* Every statement will begin with one of the acronyms ANOVA^ CORREL )r 
FACTOR to denote which linear model solution is required. A colon will follow 
the acronym. 

The following simple rules are useful.* 

Rule 6 . An equal sign will be used to separate hypothesis parameters from 
error parameters of an analysis. Hypothesis terms will be to the left^ error 
tiirms to the right of the equal signs. 

pii~] p Y . Commas will separate different tests in a model statement and a period 
will end the staccment. 

Rule 6 . The letter W will designate the constant term of the model or grand 
mean of the data when used singly as a hypothesis term. 

^Since une eigenvector's other basis of (5a) and (5b) form a basis for the 
vector spaces involved^ any of the space must be a linear combination of the 
eigenvectors. Thus^ any factor analytic solution must be expressible in terms 
of rotated eigenvectors of either (5a) or (5b). 
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Rule 9 « The number 0 when used in conjunction with either a hypothesis term 
or an error term vrill designate a set of parameters included in a model but 
not tested. 

Rule 10 . The statistical estimate of errors is assumed to be the residual 
variance of the error variables after all hypothesis effects have been removed, 
unless otherwise indicated. 

With these 10 rules it is possible to describe symbolically many of the 
common orthogonal designs. 

Example (l) Factorial analysis of variance, multivariate or univariate, two 
factors. 

M0VA:W=0,A-V,B=V,AB=V. 

W=0 indicates that the effect of the grand mean of the data is to be 
extract'll but not tested. A=V indicates a test of the variables V over the 
main effect A. B=V indicates a test for the main effect B. AB=V indicates 
a test of the interaction effect. 

Example (2) Canonical, multiple or product -moment correlation within a single 
sample. 

C0RREL:W-0,V1=V2. 

The effect of the variable means is extracted but not tested. The vari- 
ables in VI are to be correlated with those in V2 and the correlation tested 
using V2 for errors. 

Example (5) Factorial analysis of variance with partitioning in the A factor 
which is carried into the interaction. 

AN0VA:W=0,A1=V,A2=V,B=V,A1B=V,A2B=V. 



- 7 - 



Example (4) Randomized Blocks (Snedecor, 1956, p. 300 et seq . ) . 
M0VA:W=0,B=0,T=V. 

The effect B=0 is the effect of blocks (normally not tested) and T=V is 
the effect of treatments within blocks, pooled across blocks. Alternative 
representations of this design are available. 

Example (5) Subjects by Treatments designs. 

AN0VA:W=0,S=0,T=V. 

S denotes the Subject effect, T the Treatment effect which is tested. 
Alternative representations of this design are available. 

Example (6) Correlation from multiple samples. 

GORREL ;W=0, S=0, Vl=V2. 

The effect parameters S are included to eliminate the differences among 
sample means which would otherwise confound the test of the correlation between 
VI and V2 (unfortunately this is not common practice). 

Example (7) Factor analysis from a single sample. 

FACTOR :W=0,0=V. 

The variables V are the ones to be analyzed. Specification of type of 
factor analysis will not be discussed here. 

Example (8) Factor analysis from multiple samples. 

FACTOR :W=0,A=0,0=V. 

The effect A is included to eliminate the differences among sample means 
which might otherwise give rise to spurious and unidentifiable factors (unfor- 
tunately this is not common practice). 

Rule 11 . Analysis of covariance is denoted in the error term by a slash, / , 
followed by an upper case Roman letter to denote which parameters are to be 
used as covariates. When the covariates are the same for all tests in the 
model the designation may follow the acronym and precede the colon. 
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Example (9) Analysis of covariance in a factorial design. 

ANOVA :W=0 , A=Vl/v2, B=Vl/v2, AB=Vl/v2, or 
AN0 VA/v 2 :W=0,A=n,B=Vl,AB=Vl. 

The parameters V2 are the covariates. The attendant regression analysis 
may be described' as a correlation model. 

CORREL:W=0,A=0,B=0,AB=0,V2=V1. 

Example (lO) Partial correlation (product moment, multiple or canonical 
correlation ). 

CORREL :W=0, V3=V1/V2. 

V2 are the covariates for the test of the correlation between Vl and V3. 
Example (11 ) Partial correlation from multiple samples. 

CORREL : W=0 , A=0 , V3=Vl/v2 . 

Removal of the sample means from criteria (hypothesis variates), predictors 
(error variates ) and covariates is accomplished by including dummy parameters 
A for the sample means as in examples 6 and 8 above. 

Example (12) Homogeneity of regression in a factorial design ignoring other 
tests . 

AHOVA : W=0 , A=0, B=0 , AB=0, V2=0 , AV2=Vl, BV2=Vl, ABV2=V1 . 

The parameters AV2 are a Krone cker product of V2 and the parameters of A 
and test the homogeneity of regression over the A effect, say rows, of the 
design; BV2 tests homogeneity over the B effect, say columns; and ABV2 tests 
homogeneity in the AB interaction. Example 13 is more detailed. 

Example (13) A complete study of analysis of covariance (Snedecor, 1956, 
p. 39^ et seq . ). 

Let A be the main effect (states), Vl the variate and V2 the covariate. 
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ANOVA : W=0 , A=0 , V2=0 , AV2=Vl . 

The error term for this homogeneity of regression problem is the residual 
after fitting individual regressions for each sample of A. W=0 and A=0 indicate 
parameters of an analysis of variance which are to be extracted independently of 
tests of the covariate regressions. V2=0 indicates that the "common" (custom- 
arily the "error") regression effect is to be eliminated before testing for 
homogeneity of regression. AV2=Vl test the homogeneity of regression effects 
over the factor A by obtaining the differences between regressions for each 
sample after the common regression has been removed . 

This example shows the similarity of covariance analysis to analysis of 
variance by way of treating V2 as a factor of the design and AV2 as an interac- 
tion effect. 

ANOVA : W=0, A=Vl/V2, A=V2. 

The test A=Vl/v2 is the standard analysis of covariance. This model 
includes A=V2 as a check of the homogeneity of covariate means over the effect 
A. (Snedecor does not include this but it is suggested by many authors.) The 
test A=Vl/V2 is the standard analysis of covariance. The error term here is 
VI residual to A and the common regression of V2 on Vl assuming the test AV2= 

VI to be null. 

The "common" regression analysis, assuming homogeneity of regression, is 
formed as in example 10 and pools the individual regression coefficients of the 
samples. 

Method of Solution and Nonorthogonal Data 

Two types of solutions exist for linear models. These might be termed 
the part solution and the partial solution as in correlation theory. The dis- 
tinction between the two models arises when there are two sets of hypothesis 
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parameters in a nonorthogonal analysis of variance to be tested separately 
against the same set of error parameters. 

To distinguish between the trwo solutions let us consider the model: 

A=V, B=V. The partial solution tests B against V with the effects of A removed 

from both B and V, then tests A against V with the effects of B removed from 

both A and V. The part solution considers the order of the hypotheses and 
first tests B against V with the effects of A removed from both B and Y, then 
tests A against V with B removed from V only . If A and B are uncorrelated, as 
the effect parameters of analysis of variance are with orthogonal data, the two 
models are identical because A and B are uncorrelated. When A and B are cor- 
related, as in correlation or nonorthogonal analysis of variance, the test of 
B is the same for both solutions but the test of A is confounded with the test 
of B in the part solution since the variance of B which is correlated with the 
variance of A is pooled with the variance of A. Without describing the pros 
arid cons of the trwo solutions it must be noted that if the test of B is sig- 
nificant in the part solution, no "exact" test of A can be made since A is 

confounded with B. 

In computer programs it is desirable to use the part solution rather than 
the partial solution because the partial solution can be obtained from the part 
solution by simply reordering the variables as follows. Suppose we are forced 
to use the part solution for example 1. 

AN0VA:W=O,A=V,B=V,AB=V. 

The only test which has the desired partial is AB=V. To obtain all the 
tests as partials we should use all the models 

AN0VA;W=O,A=0,B=0,AB=V, 

AN0VA:W=O,A=0,AB=0,B=V, 
and AN0VA;W=O,B=0,AB=0,A=V. 
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For a more formal discussion of the part method see Bock (1965). The 
partial method is discussed by Yates (1955). 

When computer programs use the partial solution, it is impossible to 
obtain a part solution. Therefore, in this logic the part solution will be 
the rule. 

Rule 12 . The solution of the model is by the part method. The hypothesis 
of any test will be taken residual to aU. hypotheses to the left of it and will 
possibly be confounded with all hypotheses to the right of it. All errors are 
residual to all hypotheses and confounded with each other. The order of test- 
ing proceeds from left to right. 

This rule allows some alternative statements for various models. For 
instance, a one-way analysis of covariance can be stated in various ways. 

Proof of the identity is presented in a later section. 

ANOVA/V2:W=0,A=V1. 

ANOVA :W=0, A=V1/V2 . 

AWOVA:W=0,V2=0,A=V1. 

ANOVA:V2=0,W=0,A=V1. 

With this rule, the usual orthogonality restrictions on examples 1 to 15 
can be lifted without loss of meaning. 

Nested Designs 

A nested design can also be viewed as a special kind of factorial. Scheffe 
(1959; P* 179) does this with some ease. If one views the primary sampling as 

ijh 

the rows of a factorial design, then subsampling of the i primary sample j 

"th 

times produces j subsamples as the colvimn entries of the i row. For the 
simple case of three primary samples with two subsamples each, the following 
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two diagrams may be drawn to i3J.ustrate nesting. The first is a common method 
of diagramming nested designs, the second a variation which shows the similar- 
ity to factorials more clearly. 




Statistical testing involves testing the differences of means between 
and Bg, the differences in means between B^ and the differences in means 
between and B^ and the differences among the means for and 

primary samples. Using the common notation of W to denote "within" we can 
promulgate the following rule. 

Rule 13 « Nesting of several subsamples, say B, within a larger sample A shall 
be denoted with an upper case Roman W as in BWA. When several samples are 
involved in A the notation for the B effect within sample A^ will be denoted 
BWA5. 

Example (l4) The nesting design cited above. 

ANO VA : W=0 , BWA1=V, BWA2=V , BWA5=V, A=V. 



Pooling of Parameters 

Pooling effects is a convenience to have in some analytic problems. 

For instance, in some conplex factorial designs, some statisticians pool high 
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order interactions and test them collectively rather than singly. Such pro- 
cedures may be allowed for here as follows. 

Rule l4. The parameters for two effects may be pooled and tested collectively 
by placing a + (plus sign) between them. In a given level of nesting; when 
all nested effects are to be pooled before testing; the notation may be 
shortened by dropping the integers on the levels of the effect in which the 
nesting occurs. 

Example ( 15 ) A four factor analysis of variance pooling tnird and fourth 
order interactions. 

ANOVA:W=0;A=V;B=V;C=V;D=V;AB=V;AC=V;AD=V;BC=V;BD=V;CD=V; 

ABC+ ABrH-ACD+BCD+ABCD=V . 

Example (I 6 ) A nested design with two primary samples and subsamples nested 
in each (Winer, 1962 , pp. l84-l85) (fixed effects). 
AN0VA:W=0;BWA1=V;BWA2=V;A=V (B effects not pooled), 
or ANOVA:W=0;BWA1+BWA2=V;A=V (B effects pooled), 

or AN 0 VA:W=O,BWA=V,A=V (short version equivalent to B effects pooled). 
Example (l?) A complicated nesting design, partially factorial. 

This is the same design as Example I 6 with each subsample divided into 
two categories, C, which are consistent over samples and subsamples (fixed 
effects) (Winer, I 962 , p. I 86 ). 

ANO VA : W=0 , BWA= V, C =V , A=V , BCWA=V . 

Example ( 18 ) Pooling of error variables. 

AN0VA:W=0,A=V1,B=V1+V2,AB=V1. 
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Components of Variance Models 

There are two assumptions implicitly in the above rules which restrict 
the application of this logic to factorial designs. They are: (a) the error 

term for a test is the residual sum of squares remaining after all hypothesis 
effects have been removed from the da'ca, and (b) the statement of the hypoth- 
esis in say A=V assumes that a hypothesis sum of squares is generated for the 
regression of V onto A. 

These assumptions make it difficult to express a components of variance 
model in the logic. To include components of variance models in the logic 
it is necessary to invent a notation for generating a regression sum of 
squares as the error term of an analysis of variance. 

Rule 15. When an error term is to be obtained as a sum of squares due to 
regression, an asterisk, *, will be used to separate the dummy parameters and 
the continuous variables which denote the regression. 

Example (I9) A subjects by treatments analysis. 

ANOVA : W=0 , S =0 ,T=ST*V . 

This is an alternative representation of Example 5« Example 5 "t^he 
error term for the treatment effect was obtained as a residual after eliminat- 
ing the grand mean, subjects and treatments effect. Here the error term is 
generated directly as an interaction sum of squares. This intsraction sum of 
squares as a hypothesis would have been generated from a statement ST=V. 

Example (20) A complete three factor components of variance model (Kempthorne, 
1952, p. no et seq . ). Only the tests of second order interactions are 
shown assuming the third order interaction to be the appropriate error 
term. 

ANOVA : AB=ABC*V , AC=ABC*V, BC=ABC*V . 



-15- 



Note that it is not necessary to eliminate the grand mean or the main 
effects from the model since the error term is generated directly and not 
obtained as a residual. 

Assuming that the second order interactions test null we can use other 
choice of error terms for various main effects. 

ANOVA : A=ABC+AB*V, B=ABC+ABf BC*V , C=ABC*V . 

Whether these choices are rational or not is irrelevant to the notation. 
Example (2l) A nested design with four levels of nesting and replication 
(Kempthorne, 1952, p. lOf). Equal numbers of subsamples are assumed 
within each sample. 

ANOVA :W=0 , A=BWA*V, B=CWAB*V , C=DWABC*V, D=V . 

Example (22) A nested design with two levels of nesting and unequal numbers 
of subsamples. Let A be two primary samples; A1 having two subsamples, 

A2 having four subsamples. To do this it will be necessary to treat the 
design as a full 2x4 factorial with some missing cells A^ B^ and A^ Bj^. 

The B effect will be partitioned: Bl will have one parameter and B2 will 

have two. 

M0VA:W=0 , A=BlWAl+BlWA2fB2WA2*V,BlWAl=V, B1+B2WA2=V . 

Shortened, notation can be 

ANOVA : W=0 , A=B 1WA1+BWA2*V, BIWA 1=V, BWA2=V. 

Extension 

The notion of extension in factor analysis has been generalized to other 
forms of the linear model by Hall (19^9). Briefly, the technique explores the 
relationship of an external variable to the canonical variables of a significance 
test. 
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Rule l6 . When a significance test is to be extended to variables not 
involved in the test, an ampersand, &, will be used to separate the error 
variables from the variables of the extension. 

Example (23) A multiple group discriminant analysis extended to a set of 
variables, V2. 

ANOVA :W=0,A=V1&V2. 

Some Miscellaneous Examples 

These examples are included for whatever interest the reader may have 
in them. 

Example (2U) Two factor experiment with repeated measures on one factor 

(Winer, I962, p. 302 et seq . ). Two groups of subjects, S, are subjected 
to two treatments A^ and A^. Each group is given three successive treat- 
ments B-, B- and B,. 

12 3 

ANOVA :W=0,A=SWA,B=SWAB,AB=SWAB. 

Example (25) Three schools are participating in an experiment on teaching 

fourth grade arithmetic. Each school has four classes, two teachers in 
method 1 and two teachers in method 2. Differences concerned method, 
teacher and school. All effects are fixed; all students and teachers 
assigned at random within schools. Third grade arithmetic scores are 
assigned as covariates. 

(a) Check of whether covariate scores (V2) are randomly distributed 
among teacher (t), method (M) and school (S). 

ANOVA :W=0, S=V2, TWMS=V2,MWS=V2. 

The effect S=V2 would likely be significant and will be taken to be so 
here. 
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(b) Check the homogeneity of regression of third grade scores against 
final foiarth grade scores vithin teacher and method for each school. 

ANOVA ;W=0, S=0,TWMS=0, V2=0 , TV2WMS1=V1,TV2WMS2=V1,TV2WMS5=V1, MV2WS1=V1, 

MV2WS 2=V1, MV2WS 5=V1 . 

It -will be assumed that all these tests "will be null. 

(c) Check for homogeneity of regression among schools. 
AN0VA;W=0,S=0,TWS=0,MWS=0,TMWS=0,V2=0,SV2=V1. 

If this test is significant, the experiment must go to an analysis of gains 
of the form V1-PV2 where the p is different for each school. The model 
follows from example of analysis of covariance. Where V2WS removes the 
separate school regressions, the appropriate statement is: 

ANOVA :W=0, S=0,V2WS=V1,TWMS=V1,MWS=V1. 

MWS has three degrees of freedom; if this is not significant there is no 
effect due to method. If it is significant it is entirely possible that 
the significant differences occurred in only one school, out this has not 
been examined. The differences in separate schools can be discovered as 
follows . 

ANOVA :W+S+ V2WS+TWMS=0 , MWS L=VL, MWS 2=V1, MWS 5=V1 . 

The Rules as an Operator Calculus on a Set of Vectors 

Consider a set, 0,v of K vectors (the variables) in an N dimensional (the 
observations) vector space; K < N. 

Notation . Subspaces of 0 may be designated by letters of the alphabet, 
not W. Subsets of subsets are designated by a letter followed by an indexing 
digit (examples: A, B, Al, A2, etc.). The null space will be noted as <J> (zero) 

W used alone designates the unit vector. 
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Operator 1. Direct Product . The direct product of two vectors is a vector 

whose elements are the pairwise products of corresponding elements of the 
two vectors. The direct product of two subsets of vectors consists of all 
possible direct products of two vectors one from each of the subsets. This 
will be indicated by juxtaposing the two subspace notations without inter- 
vening symbols, i.e., AB, AlB, ABC (generating interaction parameters from 
main effects. This is an alternative to the Kronecker product). 

Some Algebraic Properties of the Notation 

One of the basic properties of mathematics is that it provides a simple 
notation for representing a complex event. One of the advantages of mathematics 
is that the simplified notation can be manipulated to express relationships 
among complex events when these relationships are too complicated to be observed 
directly. For example, a sheepherder can merge two counted flocks of sheep and 
know the size of the resulting flock without counting it: the mathematics of 

addition relieves him of the necessity of counting the merged flock. 

If this use of mathematics is accepted, the present notation scheme can 
be manipulated to show certain properties of linear models. For example, it is 
possible to show that some of the model statements made above are identical 
without going through the labor of calculating the model to find identical 
results. Consider the following two theorems and a lemma about identities. 

Lemma: A given model statement produces a unique ordered series of hypoth- 

! 

eses as subspaces of a vector space, when the hypotheses are correlated. 

I 

Proof: [The proof here is very simple and depends on the method of 

solution stated in Rule 12.] 
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Suppose we are given a linear model statement with hypotheses parameters 
and error parameters ...t. Rule 12 says that the hypothesis of 
any test will be taken residual to all hypotheses to the left of it and 
(possibly) confounded with any hypothesis to the right of it. This makes it 
possible to write the hypotheses of a model statement as an ordered list of 
subspaces of a vector space: a, 3,... 7 . 

It is apparent on the face of it that is a different statement 

from . . .7 when ol and 3 are correlated since in OL,^,.,,y the hypothesis ol 
is confounded with 3 in the solution, but in 3,(7, ...7 the hypothesis OL will be 
residual to p in the solution, Q.E.D. 

Example (26) Consider the alternative models 

(a) AN0VA:W=0,A=V,B=V,AB=V 

and 

(b) AN0VA:W=0,B=V,A=V,AB=V. 

If the design is not orthogonal (i.e., all cell frequencies are not the 
same), the A pseudovar tables are correlated with the B pseudovariables. 

If we follow the method of the lemma, these give rise to two ordered 
series of hypothesis subspaces (a) W, A, B, AB and (b) W, B, A, AB and are 
obviously not the same model. 

Theorem: The following statements of a one-way analysis of covariance 

produce identical analyses : 

(a) AN0VA:W=0,A=V1/V2. 

(b) ANOVA:W=0,V2=0,A=V1. 

(c) AN0VA:V2=0,W=0,A=V1. 

(d) AN0 VA/v2;W=0,V1=V2. 

Proof: Statement (d) is equivalent to (a) by definition in Rule 11. 
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statement (a) gives rise to an ordered series of hypotheses 
(a* ) W, V2, A; VI 

since the analysis of covariance is a partial correlation technique with both 
A and VI residual to the covariate. 

Statement (b) gives rise to the ordered series of hypotheses 

(b* ) W, ' V2, A; VI 

which is the same as (a' ) above and therefore produces an analysis identical 
to (a) and (d). 

Statement (c) gives rise to the ordered series of hypotheses 
(c*) V2, W, A; VI. 

Since W and V2_,are hypothesis statements to the left of A and VI, A and 
VI are residual to them regardless of what order they have between them. 

Since A=V1 is the only test being made (c) is equivalent to (b) and also 

(a) and (d). Q.E.D. < 

Theorem: The following statements about a subjects by treatments design 

are equivalent: 

(a) AN0VA:W=O,S=0,T=V. 

(b) ANOVA:T=ST*V. 

Proof: [Subjects by treatments designs are customarily designs where a 

number of subjects are given a fixed number of treatments, all in the same 
sequence. ] 

Consider V to be univariate. If the number of subjects is s and the 
number of treatments is t, then there are s*t = 1 + (s-l) + (t-l) + (s-l). 
(t-l) observations on subjects and degrees of freedom in the design. 

p 

Using a semicolon to separate hypotheses from errors. 




22 



- 21 - 



statement (a) gives rise to the ordered series of parameters W, S, T;V 
where V is residual to S, T/ and W. Since W has one parameter, S has s-1 and 
T has t-1 parameters, (s-l)*(t-l) degrees of freedom are residual in V. 

By Rule 15 one can see that ST of statement (b) is a set of (s-l)*(t-l) 
dummy parameters which can be handled as if they were a hypothesis set. ST*V 
will generate a sum of squares with (s-l)* (t-l) degrees of freedom. 

It is customary to use contrasts for S and T which are orthogonal to W 
and to each other. Thus (s-l)* (t-l) parameters of ST are also independent of 
W, S and T and, trivially, residual to them. From this we conclude that the 
sum of squares for ST*V has (s-l)* (t-l) parameters. Therefore we claim that 
V of statement (a) is identical to ST^V of statement (b). 

Now; in statement A, since T is independent of W and S the parameters T 

/ 

/ 

in (a) are identical to those of T in (b). 

Therefore both hypothesis tests give the same results. Q.E.D. 

In addition to identity relationships between model statements there are 
other types of simple relationships among models. Consider the follon/^ing 
definition. 

Definition: Two models are said to be concordant if there exists an 

isomorphism between the subspaces of each. 

This requires that there be a one to one correspondence between; (l) the 
ordered series of hypothesis subspaces of the two model statements and (2) the 
errors subspaces of the two model statements, in such a way as to preserve the 
mathematical relations between hypotheses and errors within both statements. 
This does not require that the vectors of the subspaces be related in any way. 

For example, the two model statements 

AN0VA:W=0,A=V,B=V,AB=V. (a) 
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and 

COREEL:W=0,V2=V1,V5=V1,V5=V1,V2V5=V1 (b ) 

are concordant if we set up the correspondence 

model (a) W f+W model (b) 

A V2 
B V5 

V VI 

This correspondence must also extend to the mathematical operations of 
testing and Kronecker product. These models satisfy the definition of con- 
cordance because 

model (a) AB V2V5 model (b) 

W=C <^W=0 
A=V V2=V1 
B=V V5=V1 

and AB=V 4 ^ V2V5=V1 

In this example the ANOVA model is a two-factor factorial analysis of 
variance, the CORREL model is a study of response s\irfaces. 

The model 

AN0VA;W=0, A=Vl, V2=Vl, AV2=V1 (c) 

is also in this concordance class since (c) we can set up the correspondence 

model (a) W 4+W model (c) 

A 4 ^ A 
B 4^ V2 

V 4 ^ VI 

Discussion 

It sho\ild be noted that, throughout the description of the notational 
scheme and its logical properties, there is no discussion of either the 
distributional form of the data or whether the models have any exact test in 
any distribution. Thus, it is up to the user to state the nature of data 
distributions and to determine whether an exact test is available. For 
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instajice, while components of variance models can be stated for nonorthogonal 
models there is no assurance that a solution exists which can be tested as an 
exact test. 

A similar but less complex scheme was produced by Cramer (see Clyde 
et al ., 1966) for use with a program to compute multivariate analysis of 
variance. The instigation to develop this system came from Cramer's work 
and the need for a control mechanism for a similar type of computer program 
for linear models . 
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